Working branch #20402
Closed
Working branch #20402
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Alexander Jipa <[email protected]>
…#19727) Co-authored-by: dominicgkerr <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jirka Borovec <[email protected]>
…AI#19690) Co-authored-by: Sebastian Raschka <[email protected]>
* update * update
…19852) * distributed checkpoints * use decorator * refactor if-strict * update example * filter non-persistent buffers (todo, add test) * simplify checkpoint loading for model
…oints (Lightning-AI#19870) * memory-optimized loading of full checkpoints into dist model * simplify * handle buffers * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * handle strict loading, buffers, and add test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chlog --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…ghtning-AI#19872) * Load optimizer state * move to utility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* ModelParallelStrategy for Lightning Trainer * mypy * import fix * fix torchscript errors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix docs issue * fix test execution * Update src/lightning/pytorch/strategies/model_parallel.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Luca Antiga <[email protected]>
* Add 2D parallel example * replace with torchtitan code
* Add documentation note for TQDMProgressBar --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* upggrade requiremnets.txt * update fabric bitsandbytes linear quantization for bnb 0.44.1 * add quant_storage param * exclude macos from bnb upgrade * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…g-AI#20325) Co-authored-by: peter.mcelroy <[email protected]>
* docs: fix removed ref to `deepspeed.initialize` * fix links
…ning-AI#20354) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Borda <[email protected]> Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Jirka Borovec <[email protected]>
update tutorials to `b83fde09` Co-authored-by: Borda <[email protected]>
pmatczak2
requested review from
Borda,
ethanwharris,
justusschock,
lantiga,
tchaton and
williamFalcon
as code owners
github-actions
bot
added
docs
Collaborator
|
Thank you @pmatczak2 , your PR targets the I'll close it for the time being:
thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
43 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added detailed comments throughout the _FitLoop class to explain the purpose and functionality of key methods, particularly setup_data(), on_run_start(), and on_advance_start().
Clarified Logic for Data Loading:
Explained the rationale behind multiple calls to setup_data(), emphasizing its role in ensuring that data loaders are fresh for each epoch and the conditions under which they are reloaded.
General improvements to code readability by adding comments that provide context for the flow of the code and the design decisions made, making it easier for future developers to understand the implementation.
setup_data(): Documented its purpose in managing training data loaders and handling overfitting scenarios.
on_run_start(): Clarified its role in setting up validation data loaders and invoking relevant hooks.
on_advance_start(): Explained the necessity of calling setup_data() to prepare for the current epoch.
📚 Documentation preview 📚: https://pytorch-lightning--20402.org.readthedocs.build/en/20402/